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This article describes the process by which one study utilized queditative 
methods to create items for a multi dimensioned scale to measure twelve 
step program affiliation. The process included interviewing fourteen 
addicted persons while in twelve step focused treatment about specific 
“pros” (things they like or would miss out on by not being involved in 
twelve-step programs ) and “cons” (things they dislike or would benefit 
from if they did not engage in twelve-step programs). The triangular 
process used in qualitative research is described, which generated items 
for the subsequent instrument to measure ambivalence toward recovery 
programs. Mixed-method strategies included qualitative interi’iewing to 
inform scale development and three analytical approaches to produce 
specific codes, themes, and domains. Key Words: Mixed Method 
Research, Sccde Development, and Twelve Step Programs 


Introduction 

Padgett (1998) and Weiss (1994) describe a rationale for the use of qualitative 
interviewing to provide preparation for quantitative studies. This qualitative preparation 
is often conducted for survey research. By conducting qualitative interviews prior to 
surveys, key information from participants in specific social/behavioral circumstances 
(e.g., addicted individuals in twelve step recovery programs) can enrich the quality of the 
research. Analyzing data generated from the interviews informs the survey designed for 
larger samples. Furthermore, analysis of data from surveys can be analyzed from either or 
both a quantitative or qualitative approach. This broad mixed-method tradition provides 
the foundation for the description of this particular study. The next few paragraphs 
describe the background of this study within the context of substance abuse research and 
the focus on informing the development of a scale to measure ambivalence toward twelve 
step recovery programs. 

Generally true in substance abuse research, little is reported about how items for 
instruments, checklists, or inventories are generated. For example, Baker, Sellman, and 
Horn (2001) report on the construction of the Attribution to God’s Influence Scale 
(AGIS), which observes alteration in perceptions of God’s influence of people involved 
in twelve- step recovery. They mention constructing items based upon consultations with 
colleagues and participants in spirituality classes, but a specific description of how these 
consultations resulted in the generated items was not included. Given the lack of attention 
about item generation in the psychometric literature, one might conclude that the topic is 
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not an important issue/concern. From a quantitative or statistical point-of-view, the 
origins of questionnaire items are not significant. The key is whether or not the items 
represent the construct or variable in question as measured by reliability and validity 
scores; not where the items came from. 

Padgett (1998) mentions a multimethod combination depicted as qualitative to 
inform quantitative efforts in developing scales. In this method, the qualitative study 
comes first and is used to explore concepts and to identify hypotheses. Using qualitative 
inquiry can be especially useful to researchers in the development of scales. In essence, 
validity of concepts and inquiries in quantitative research can be enhanced by first being 
grounded in real life situations and observations through having conversations or 
interviews from an open perspective. 

In addition to psychometric concerns, where the items were first located and how 
they were shaped or edited, provides an important context that reveals assumptions and 
theoretical positions of the authors of those items, highlighting what domains of 
knowledge or expertise they privileged as well as those domains that were omitted. 
Examining these early stages may also serve as a vehicle for us to see if there may be 
some yet unexplored or untapped areas of the topic in question (or new informants) that 
could yield specific new items or entire new contexts for questions. 

Making transparent the early stages of item development also provides valuable 
insight for those who seek knowledge, skills, or experience in the process of scale 
development. The process outlined in this article may also provide an exemplar for others 
who are looking for templates or guidance on how to develop their own measurement 
items in their chosen field of interest. Describing this process in detail highlights the 
advantages and disadvantages of different methods of originating questions for item 
development. Revealing the processes by which scale items are first located and then 
refined, grounds those items in a more inclusive context that may add to the confidence 
we place in those items. In scale development, the primary goal is to create rigorous 
scales that reliably and validly represent the best possible questions we can devise. 

In qualitative research, disclosure of choices made by the researcher(s) and the 
thinking involved in making those choices are essential in clarifying the assumptions and 
theoretical/philosophical dimensions of the methodology (Denzin & Lincoln, 2000; 
Patton, 2002). Methods, protocols, and measurement devices emanate from preferred 
understandings of the world and of what constitutes knowledge. Qualitative research 
understands the research enterprise as occurring within a context (e.g., academy, 
organization, governmental agency, industry) that significantly shapes the research itself 
(Greenwood & Levin, 2000). Making the context explicit aids the reader in better 
understanding the findings and how they make sense, and to see and understand the 
larger systems/forces within which the study itself is situated. 

Going through a process of outlining and describing these choices and 
motivations assists the researcher to better comprehend her/his own process as a 
researcher, improving the current project’s focus and rigor, as well as sharpening the 
researcher’s item generation skills in terms of other future research projects. Building in 
systems of “unpacking” or examining preferences of all choices made in the formal 
research process (as well as in the earlier pre-formal stages), enhances the researcher’s 
understanding of the validity of her/his efforts and to what extent it may be biased in one 
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direction or another. Only a careful reflection on each research decision/step will enhance 
confidence in the effort to come to know authentic lived process(es). 

Study Domain of Interest 

Formal alcohol and other drug treatment programs utilize twelve- step programs as 
free and accessible adjuncts that are consistently associated with improved substance 
abuse outcomes (Emrick, Tonigan, Montgomery, & Little, 1993; Tonigan, Connors, & 
Miller, 2003; Tonigan, Miller, & Connors, 2000). Hence, it is considered by many to be 
standard practice for treatment providers to rely on twelve- step programs to supplement 
treatment and as a primary source of post-treatment support (Borkman, Kaskutas, Room, 
Bryan, & Barrows, 1998; Humphreys, 1997). 

Even with the evidence of improved outcomes associated with consistent 
involvement in twelve-step programs, it has also been demonstrated that there is a high 
rate of dropout; estimated at 50% within 90 days and 90% by the end of the first year 
following their initial experience with Alcoholics Anonymous (Miller & McCrady, 
1993). The Project Match Research Group (1997) observed this high rate of dropout and 
less than regular weekly attendance (sometimes referred to as disaffiliation) among those 
who were treated in twelve- step facilitative methods in the year following treatment 
(Tonigan et al., 2003). Research evidence suggests that minimal and sporadic twelve-step 
program involvement or complete noninvolvement places those clients at greater risk for 
resumption of the problematic behaviors. Though research has demonstrated that 
treatment can increase post-treatment affiliation with twelve- step programs, little is 
known about specific factors that discourage affiliation. 

This concern has stimulated an increasing interest in research focused on 
identifying factors that could predict disaffiliation in order to better inform treatment 
providers who may be able to adjust treatment procedures and methods so as to reduce 
disaffiliation (Connors, Tonigan, & Miller, 2001; Fiorentine & Hillhouse, 2000; Kelly & 
Moos, 2003; Mankowski, Humphreys, & Moos, 2001; Tonigan et al., 2003). To date, no 
research has provided a model of factors and influences that can reliably explain the 
process of twelve-step program affiliation (or disaffiliation). Also, no research was found 
that directly seeks the ideas of newly sober individuals about pros and cons of twelve- 
step program involvement. Thus, this study could contribute to the literature by directly 
tapping into the perceptions of alcoholics, concerning why they choose to affiliate or 
disaffiliate with Alcoholics Anonymous in order to better inform items for a scale to 
measure ambivalence in affiliation. 

Researcher Collaboration 

This project was undertaken by three colleagues in the Kent School of Social 
Work at the University of Louisville subsequent to official approval given by the 
Institutional Review Board. Noell Rowan was a graduate research assistant and a full- 
time doctoral student in the Kent School at the time that this study was conducted. Her 
background is in clinical social work with specializations in substance abuse and 
chemical dependency treatment as well as gerontology and general mental health 
practice. She was interested in learning more about both qualitative and quantitative 
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research methods, and saw this project as an opportunity to learn about both of these 
approaches with two professors who have significant experiences in each methodology. 
She is now an assistant research professor at the University of Louisville’s Kent School 
of Social Work. 

When the study was conducted Dan Wulff was an associate professor at the 
University of Louisville’s Kent School of Social Work. He is a qualitative researcher 
with a background in marriage and family therapy. Dan is a co-editor of an online 
qualitative research journal and teaches qualitative inquiry in the doctoral research 
sequence. 

Rick Cloud is an assistant professor at the University of Louisville’s Kent School 
of Social Work. His primary research interest in improving community based alcohol and 
other drug treatment outcomes led him to initiate this project. Though his research 
experience was in quantitative methods, he was interested in the potential of using 
qualitative research to obtain a more in-depth understanding of affiliation or disaffiliation 
factors. 


Steps 

Originally, this project was conceived by Rick in order to develop a list of pros, 
cons, and expectations of twelve-step programs from people in treatment settings (the 
outcomes of this project are reported in Cloud, Rowan, Wulff, & Golder, in press). Noell 
was given the opportunity to join Rick in this endeavor due to her similar research 
interests. Rick used a theoretical model (based upon Motivational Inten’iewing), which 
focused on the belief that ambivalence is a factor influencing motivation and engagement 
in twelve-step programs (Miller & Rollnick, 2002). This motivational theory could be 
used to explain why clients do not attend twelve-step programs as prescribed (weekly or 
more) after treatment. More specifically, these theories would point to ambivalence 
surrounding attendance at twelve- step meetings as being central in creating their 
disaffiliation. Ambivalence is characterized as the pros (things people like or would miss 
if they did not attend) contrasted with the weight of the cons (things they dislike or would 
gain if they did not attend). Miller and Rollnick suggest that the balance of the pros 
compared with the cons will create ambivalence and increase the tendency to disaffiliate. 
Originally developed for use in motivating changes in alcohol abuse, Motivational 
Inten’iewing has been successfully adapted to explain non-compliance with a wide 
variety of health behaviors (e.g., smoking, abuse of drugs other than alcohol, diet, 
exercise, medication compliance, treatment session attendance, HIV risk behaviors). 

For Rick and Noell, the initial task involved compiling a detailed list of pros and 
cons to affiliation/disaffiliation. In order to more concisely capture ambivalence, the list 
of pros and cons would need to be comprehensive and sensitive to the range/variety of 
factors involved in decisions to affiliate/disaffiliate. The purpose of developing this 
composite list of pros and cons was to better understand why people affiliate/disaffiliate 
and to create a scale which could be used by counselors to assess the likelihood of post- 
treatment dropout, and, as a result, to make adjustments if possible that could reduce 
dropout, or alternatively, to offer the newly recovering person some other form of 
aftercare. Rick and Noell initially planned to use a quantitative survey with a sample of 
people in treatment to obtain this information. A decision was made to conduct a 
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qualitative inquiry to prepare for the survey based on discussions with colleagues and 
review of the literature about mixed method approaches (Monette, Sullivan, & DeJong, 
2002; Padgett, 1998). In an effort to produce more detailed and varied responses, they 
elected to use a qualitative set of interviews to garner this information about 
affiliation/disaffiliation. 

To build in the qualitative interviews, Rick then approached Dan about joining 
this research team. While Dan preferred a more open ended qualitative interview, Rick 
remained committed to a semi- structured set of questions due to his perception that an 
open ended study would be too time consuming. A compromise was reached whereby a 
semi-structured interview protocol was developed, while including opportunities for the 
respondents to expand on their responses whenever the interviewer felt it appropriate. 
Some examples of questions from the interview protocol were: (1) What do you like 
about twelve-step programs (TSPs)? (2) What would you miss out on if you did not 
attend TSPs? (3) What do you dislike about TSPs? (4) What would you rather do instead 
of attending TSPs? 

These interviews would be conducted by Noell and Rick and the transcribed 
responses would then be coded by each of the three researchers independently of one 
another. A grounded theoretical context would be used to guide the coding process. As in 
grounded theory, it is important that the researchers learn from the abstract data in terms 
of what problems might emerge instead of forcing an agenda (Glaser, 1992). The process 
of coding would begin with open coding, where an initial constant comparative analysis 
occurs prior to naming the codes and placing them into categories (Glaser). The coding 
would be conducted from a descriptive perspective without any prior conceptions of what 
the codes might be. The terms codings and themes are used interchangeably as they are 
meant to describe the important codes or themes derived from the interview data. The 
three codings/themes would then be compiled by Rick, noting the differences among the 
three sets. More specifically, all three researchers were to analyze the list of pros and 
cons and compose sets of codes/themes independently of each other. Rick would then 
view each of the three lists and make note of the differences in the lists. Therefore, the 
specific qualitative methodology is based in grounded theory to create codings/themes. 
The interview information as represented in the three sets of codes/themes would provide 
the necessary voice from the interviewees. Hence, the data gathered from the interviews 
would speak through the three sets of codes and their compilation. 

Rick was familiar with what the current literature would offer as reasons for 
disaffiliation and affiliation. This information would be compared to the composite list 
that the research team would generate from the interviews, to see what the interviews 
added (if anything) beyond the already existing literature (see Cloud et al., in press, for a 
full discussion of how the literature compares to results of the study). 

The Interviews 

“To be means to communicate” highlights the deep human need to talk with 
another person (Bakhtin as cited in Gergen, 1999, p. 131). In these interviews, Noell felt 
gifted to travel with the interviewees on their road to co-construct understanding of their 
individual experiences in recovery. A unique connection was made with the interviewees 
in asking them to describe in depth their positive and negative experiences with twelve- 



455 


The Qualitative Report September 2007 


step meetings; questions not typically asked of addicted people who are engaged in a 
formal twelve-step focused treatment or recovery process. Perhaps the reasoning for this 
is that in chemical addiction treatment there is typically a heavy emphasis on becoming 
involved with twelve-step recovery as a life and death matter (and consequently less 
concern for likes and dislikes). 

Many of the interviewees were homeless and represented cultures different from 
the experiences and background of Noell. To keep in mind the power of listening, really 
listening, to another person (particularly persons whose voices are not usually included 
prominently in research) creates a powerful moment for the researcher of doing 
something really meaningful. Not only does the interviewer come to know many new 
points-of-view and appreciate the person in new ways, the interviewee can also have a 
validating experience of having someone (a researcher) genuinely take an interest in 
her/his viewpoint in some detail. The verbatim recording of the words of each 
interviewee as well as the use of active listening techniques assisted the interviewer in 
going deeply into the specifics of each interviewee’s story. 

The interviewees were informed that their words were going to be used in the 
development of a scale to help other at-risk clients in the future. Being able to include the 
interviewees in this scale development process was gratifying to them. Many of the 
interviewees commented on how they appreciated being asked about their experiences 
from a researcher who was going to give their ideas significant influence. (Subsequent to 
this study, the scale that these interviews helped develop was used by Noell and Rick in a 
large quantitative study. Several of the interviewees who had provided information that 
led to items for the scale were themselves given the scale and were noticeably pleased to 
see that many of their responses were included in the scale.) 

The Process Continued 

After each day of interviews conducted by Noell, Dan met with her to debrief her 
interviewing experience, looking for any instances of where she might have changed her 
interview style or process (or desired to). Debriefing was only conducted with Noell 
because Rick only conducted three interviews subsequent to Noell’ s eleven to assure a 
point of saturation. There were instances when Noell altered her style of interviewing. 
For example, she found that when she began the interview with questions about their 
dislikes of AA or other twelve-step groups, there was more discomfort and a sense of 
holding back from sharing information. However, when she began the interviews with 
questions about their likes of AA or other twelve-step groups, a sense of ease in their 
sharing of detailed information was experienced. It was then much easier to talk about 
their dislikes. 

Eventually, a point of saturation was reached wherein no new properties, 
dimensions, or other information was being contributed in the interviews (Strauss & 
Corbin, 1998). Rick then went out to another treatment facility and conducted three more 
interviews in an effort to determine if any new ideas would be generated from these 
additional interviews. No significantly different ideas came from these additional 
interviews. Saturation was then determined based solely on reflections on the interviews. 
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Description of Interviewees 

The sample of interviewees consisted of 14 newly recovering individuals enrolled 
in six treatment and recovery programs in the Louisville, Kentucky metropolitan area. 
The treatment and recovery programs were based on methods which focused on the same 
objectives of Twelve Step Facilitation (TSF; Nowinski, Baker, & Carroll, 1992). 
Purposive and convenient sampling was utilized in this study. The interviewees were 
intentionally balanced, with three being from two different outpatient programs (one was 
non-profit and the other for-profit) and six others represented non-profit inpatient long- 
term residential treatment and recovery programs. 

In the sample, there were ten males and four females. The mean age was 40 with 
nine Caucasians and five African Americans. Four reported that they were homeless, four 
had a place of their own, two had arranged transitional living, and four were moving in 
with a friend or family member. The average length of time in the current treatment 
program was 13 weeks. Most of the participants (N=10) reported annual income from all 
sources of less than $15,000 per year, two between $15,000 and $30,000 and two over 
$30,000. A majority (78% or N=ll) of the sample reported alcohol as their primary drug 
of choice, with only two reporting crack cocaine and one reporting marijuana. Other 
sample characteristics included: eight (57%) reported 12 years of education; two less than 
12 years and four reporting more than 12 years. Half of the participants identified as 
single, three reported being married or partnered, while four reported being divorced. All 
14 of the respondents reported an acceptance of religious beliefs and none reported being 
agnostic, atheist, or spiritual. Also, all participants reported that they could see 
themselves as a member of AA or NA either now or after discharge from their current 
treatment or recovery program. 

Impressions were that most of the interviewees were open about their experiences 
and their opinions about twelve- step recovery. The interviewees were easily engaged in 
the process of talking about their likes and dislikes for the most part. One of the 
participants, however, seemed to be wary of the professionals of the treatment center 
finding out about what she had to say. The written preamble consent seemed to ease her 
discomfort once reviewed several times and discussed. Once the interviewees were asked 
to talk about themselves and their experiences, most of them were very verbal and it took 
some swift handwriting and recording to keep up with their enthusiastic pace of sharing 
pertinent information. Hence, many times the interviewees were asked to slow down or to 
repeat their words as they were informed of the value and importance of recording 
verbatim each statement that they had to share. Once the likes and dislikes were written 
verbatim, the interviewees were asked to review each statement and to rank the 
importance of each statement as either very important, somewhat important, or not that 
important. 


Description of the Amount and Nature of the Data Collected 

After Noell conducted eleven interviews, a list of the participants’ 152 pros (likes) 
and 180 cons (dislikes) was derived and then sent out electronically to the other two 
researchers for their perusal. This list was created by Noell after the fourteen interviews 
were transcribed. All of the words for the list were taken verbatim from the transcribed 
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interviews. All three researchers then merged this long list by examining the list for 
repetitive statements, which created a new list with only 85 pros and 111 cons. The next 
step was to look for patterns in the data and then group the pros and cons under specific 
codes or themes. Each of the three researchers (Rick, Dan, and Noell) did this work 
independently and then met to compare their three generated lists. 

The Three Processes of Creating the Lists 

Three analytic strategies guided by grounded theory were employed with the 
interview data. We selected these strategies in an effort to analyze the data independently 
of each other and to provide more in depth analysis given the varied backgrounds of the 
three researchers. The processes of analyzing the data are described in the following 
paragraphs. Table 1 presents the three researcher approaches to analyzing this data 
(Constas, 1992). 

The process that Noell used to create a list of l ik es and dislikes, which were 
grouped under specific thematic titles, first consisted of reading the list through several 
times to get a sense of the meanings or themes underneath the actual words. More 
specifically, there were themes that emerged through an overlap of words or specific 
groupings of comments made by the interviewees. Out of this process came several 
domains or categories. Noell had a distinct advantage in this process as she was able to 
recall the conversation with the interviewee and reflect upon their intonation and 
explanation of their l ik es and dislikes. In essence, the fruitful discussions held about their 
likes and dislikes provided much in the way of an expanded understanding apart from 
mere words on paper. 

There were initially five themes which emerged for Noell. 

1. Message, which referred to the content of the message in the meetings. 

2. People, which referred to the descriptions of the people involved in twelve-step 

programs, their attitudes, personalities, and behaviors. 

3. Change, which referred to the specific changes from old behaviors and thinking 
brought about in the recovery process by obtaining skills/tools such as positive 
attitude, desire to change, learning to work the steps, personality changes, helping 
others, etc. 

4. Safe place, which referred to descriptions of the actual meeting place(s) and the 

feeling of safety and security within the program and specific meeting(s). 

5. Spirituality, which refers to the gifts and benefits of involvement with TSPs such as 

blessings, miracles, and spiritual connection(s). 

Pros by Noell 

1. The meetings have strong, powerful, and honest messages. 

2. People are welcoming in TSP’s. 

3. The people help me to not go back to drinking and using. 

4. 1 like mixing with different walks of life in TSP’s. 

5. TSP’s help me to change my life. 

6. Going to TSP’s helps me to get outside of myself and know others. 
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7. My thinking is better when I go to meetings. 

8. 1 like the spiritual environment in TSP’s. 

Cons by Noell 

1. People talk negatively in TSP’s. 

2. 1 feel like an outsider in meetings. 

3. 1 feel pressure to participate in meetings. 

4. The meetings are uncomfortable for me to attend. 

5. Meetings interfere with too many other things. 

6. The actions suggested in TSP’s are difficult for me. 

7. The meetings are too crowded. 

8. People talk about God too much in meetings. 

The process used by Dan was also to examine the data for themes. The initial data 
(the written lists provided by Noell) that Dan was given was examined for content 
similarities between items. Dan was only using the written lists provided by Noell; he had 
no background in the actual interviews (as Noell had). When similarities were found, 
those items were grouped under a label or code that seemed to fairly represent all items 
so grouped. The responses were initially reduced to 36 pros and 44 cons. If specific items 
were sufficiently different than the other items, they were given a code of their own. 
Several iterations of grouping codes (axial coding from the methodological basis of 
grounded theory) resulted in a set of themes that could not be further combined without 
losing some of the richness of the data. Six categories of pros and six categories of cons 
were then constructed based on the themes rising from the data and they were represented 
by statements. 

Pros by Dan 

1. 1 receive support or validation. 

2. 1 can help others. 

3. 1 like the positive atmosphere among the attendees. 

4. 1 appreciate the spiritual dimensions. 

5. 1 receive helpful information. 

6. The meetings help me achieve and maintain sobriety. 

Cons by Dan 

1. The physical setting is uncomfortable. 

2. Social interaction is problematic. 

3. Going to meetings is too inconvenient. 

4. 1 feel intruded upon in these meetings. 

5. 1 disagree with this overall approach. 

6. Attendees are too self-centered or uppity. 
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Rick had studied two research textbooks from teaching a foundational research 
class for MSSW students (Monette et al., 2002; Padgett, 1998) that guided his coding. 
He was particularly influenced by the coding methods advanced by Monette et al., which 
provided more specific guidance on coding methods that can be used to derive thematic 
codes. The primary objective of the coding process was to create themes that were 
exhaustive and mutually exclusive. The resulting coding process could best be described 
as an iterative process of developing codes and then testing codes for fit against new or 
different qualitative items. While most coding activity occurred at initiation, the process 
of refining final codes continued for several months and was influenced by several 
factors including codings of Dan and Noell, quantitative research literature reviewed on 
the topic of affiliation, new insights derived from repeated reading of the data and codes, 
thoughts and opinions of others who were consulted along the way, and writing a grant 
application that sought to develop a model of affiliation. The initial codings included 
separate yet similar codes for pros and cons, as follows. 

Coding Definitions for Pros 

Identification: Awareness, appreciation, understanding, openness to others that have 
shared the common problem of addiction, which has influenced and shaped their lives in 
similar ways. 

Support: Feeling accepted/welcome/esteemed/loved/cared for, receiving encouragement, 
having a forum to process significant feelings/events, receiving guidance and help. 
This is contrasted with various TSP practice designed to help people feel supported that 
would be included under “format/culture” category. 

Spirituality: Feeling connected by some transcendent being or experiencing some sense 
of meaning and purpose outside of self. Being rewarded by helping another person or a 
group. This is contrasted with TSP spiritual practice that would be included under 
“format/culture” category. 

Effective: Belief that TSP’s increase motivation to change, instill hope, reduce substance 
use, or improve biopsychosocial functioning. 

Format/Culture: Referring to established TSP values and normative practices. 

Coding Definitions for Cons 
Identification: Does not identify with group members. 

Support: Perceived or real (1) rejection by TSP members or (2) social discomfort (social 
phobic reactions). (2 themes ) 

Spirituality: Discomfort with spiritual or religious beliefs or practices of TSP. 
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Effective: Unwilling to invest and involve oneself sufficiently in AA/NA because (1) one 
cannot see significant benefit in involvement or (2) due to placing greater value on 
competing demands for time. 

Format/Culture: Dislikes TSP normative practices and values including abstinence. 

Logistics: Issues relating to physical or fundamental barriers to attending meetings (e.g., 
location, childcare, health and safety issues, handicap needs). 

Intolerance: Difficulty acculturating or mixing with different types of people. 

Table 1 


Documentational Table for the Development of Codings/Themes/Domains 


Noell 

Rick 

Dan 

1. Compile the list of pros 
and cons based on 
transcribed interview data 

2. Merge the list by 
eliminating repetitive 
statements to produce a 
smaller list 

3. Analyze the list to create 
specific codings/themes 
which may later be used as 
domains for the scale. 

4. Create a list of pro and 
con exemplar statements 
based on the themes. 

1 . Merge the list by 
eliminating repetitive 
statements to produce a 
smaller list 

2. Analyze the list to create 
specific codings/themes 
which may later be used as 
domains for the scale. 

1. Merge the list by 
eliminating repetitive 
statements to produce a 
smaller list. 

2. Analyze the list to create 
specific codings/themes 
which may later be used as 
domains for the scale. 

3. Create a list of pro and 
con exemplar statements 
based on the themes. 


3. Review the three 
researchers’ lists of 
codings/themes/and 
exemplar statements and 
construct the final list of 
seven themes. These seven 
themes became the seven 
domains for the resulting 
scale. 



The Process of Combining the Lists 

Rick’s list was similar in content to Noell’s five themes. For example, Noell’s 
“change” theme is very close to Rick’s “effective” theme. Given similarity in Rick’s pros 
and the cons codings, they were next combined into one set of codings that could exhaust 
all qualitative items, both pros and cons. To improve validity and confidence in the 
codings, Rick compared his codings to Noell’ s and Dan’s, which created some minor 
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changes in definitions, and provided some degree of confidence in validity of the 
resulting codings. This comparison occurred because Rick was most fa mi liar with the 
extant knowledge from the literature and could use that grounding to see if, and how, the 
codings from Noell and Dan were producing anything different (or not). 
Noell’s initial codes were in a comparable format as Rick’s with code names and 
definitions (see above), which made a direct comparison of code consistency possible. 
However, Dan’s codings were in a different format (i.e., an exemplar statement format, 
described above), which made direct comparison more difficult. Since Noell’s format 
was similar, Rick compared, contrasted, and considered the fit of his codings on 
qualitative items to those produced by Noell. Dan’s exemplar statements were then coded 
and evaluated for goodness of fit. Per a request to review Dan’s codes, Noell later 
provided her own set of exemplar statement codings (similar to Dan’s and described 
above) that were similarly tested for fit to Rick’s final list of codes. Noell perceived that 
it might assist the process if she coded her responses similar to Dan’s. 
Noell’s and Dan’s exemplar statement codes fit the final coding system developed and 
are reported by Cloud et al. (in press). 

According to Faul and Van Zyl (2004), the process of scale development must 
include a preliminary process of analyzing a problem enough to clearly establish a need 
for a scale. This predevelopment stage is crucial in an effort to establish the theoretical 
framework and to justify the need for a new measurement tool. Once the initial stages are 
complete, it is crucial to identify the construct to be measured in a clear, unambiguous 
manner. In this study, as stated earlier, the construct to be measured was the ambivalence 
related to involvement in twelve- step programs. The development phase is next wherein 
the following occurs: (a) the design of the particular items for the scale, (b) the scale 
length is determined, (c) scaling of the items, (d) developing a scoring formula, and (e) 
instructions are written for the respondents (Faul & Van Zyl, 2004). 
This next paragraph presents an overview of the process of converting the themes into the 
scale domains. The final seven themes were used in constructing the seven domains of 
the resulting scale. These thematic domains later summarized by Cloud et al. (in press) 
were placed into three broad categories consisting of (1) beliefs : congruence of personal 
beliefs with twelve-step recovery beliefs, values, and normative practices; (2) 
socialization-, the reaction of the participant to the social environment of twelve-step 
recovery groups; and (3) competing needs : the degree to which twelve-step program 
attendance fits with the other perceived needs. Attempts were made to eliminate any 
overlap among specific categories. The final thematic coding categories minimized the 
categorical overlap although overlap could not be completely avoided (e.g., the category 
positive expectancies included statements that described assessment of recovery group 
support). However, the broader categories of congruence with beliefs, socialization 
process factors, and competing needs appear to stand on their own as independent 
categories. 


The Resultant Scale 


The Twelve Step Ambivalence Scale (TSAS) was developed as a result of this 
qualitative research process. Rick’s seven primary themes were used to test separate 
scales measuring domains of ambivalence related to affiliation. Another manuscript has 
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been submitted for publication, which provides a detailed description of this scale and its 
psychometric properties (Cloud, Golder, Rowan, & Van Zyl, 2007). The instrument was 
developed to provide direct service providers with a scale to measure unresolved 
ambivalence and risk of underutilization of twelve-step programs. 

The TSAS provides a new beginning of exploration into the particular make-up of 
ambivalence with twelve- step recovery. The aforementioned description of literature on 
motivation theory (Miller & Rollnick, 2002) and another article written to describe the 
literature on affiliation with twelve-step recovery groups (Cloud et al., in press) provide a 
review of quantitative literature used to support a motivational- socio-cultural model 
focused on behaviors representing the process of assimilation and acculturation into 
twelve-step groups. Motivation related to specific behaviors is posited as necessary, but 
not sufficient for assimilation/acculturation; rather, engagement in the behaviors of (a) 
internalizing the norms, values, and beliefs, (b) involvement in formal or informal 
mentoring relationships, and (c) attending meetings with a sense of comfort and a 
positive attitude contributes to a process of affiliation. 

Results of this research provide support for unresolved ambivalence related to 
these three behaviors. Unresolved ambivalence was noted in the long and varied list of 
dislikes, the observation that there were far more dislikes than likes, and the emergence 
of a “competing needs” theme in the analysis that suggested a conflict with spending their 
limited time attending meetings as opposed to other demands, such as childcare or work 
responsibilities. These results emerged as new information that we had not seen in the 
literature. 

Increasing awareness of clients who have unresolved ambivalence with 
discussions of likes and dislikes of twelve-step programs is a first step toward assisting 
clients in alternative solutions to possibly avoid future relapse with chemical abuse. 
Alternative solutions can involve assisting with parenting needs, linking with a mentor 
who can help with exploring belief systems, or many other options to expand support 
networks. 


The Relative Value of this Study 

Now that the detailed qualitative process of developing items for an instrument 
has been revealed, the relative value of this process is presented. It should be noted that 
the resulting instrument has since been developed and other than one sub-scale (logistical 
needs) reliability of the instrument is outstanding (Cronbach Alpha >.87 on the six sub 
scales; see Cloud et al., 2007), suggesting that this form of item development holds 
promise. A scale is known to be reliable when the instrument has a high Cronbach Alpha 
coefficient. 

Each of the three researchers reflected upon how their involvement in the project 
was meaningful to them. This project was very important to Rick, as it met his goal of 
adding to his theoretical work about posttreatment disaffiliation from twelve-step 
programs. Rick reported that conducting this qualitative study was important to 
advancing theory of disaffiliation with twelve-step recovery groups. The lack of an a 
priori research protocol and the subjective nature of when one reaches “saturation” in 
responses were aspects of this qualitative study that Rick found problematic. Rick’s 
quantitative leanings would lead him to interview more subjects to assure all of the less 
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frequent responses and themes had emerged before stopping, although, in hindsight the 
responses he obtained fit neatly within the codes developed prior to collecting these 
additional observations, suggesting that Noell’s subjective assessment of “saturation” was 
an accurate assessment. 

The involvement that Noell had in working on this project was instrumental in her 
educational process in a myriad of ways. She learned about working with two researchers 
coming from varied approaches to research and how to glean knowledge from both ways 
of thinking and approaching a study. Her goals of learning about qualitative methods and 
being involved in pioneering addiction research were met. 

This work was valuable to Dan, as it met his goal of working with colleagues who 
were unfamiliar with the process of qualitative research. It was a hoped-for experience of 
“seeing what happens” that Dan desire; it was almost a guarantee that something 
interesting would transpire. There were struggles and differences of opinion on how to 
proceed, but the process of working out these issues and moving forward was gratifying. 

Lessons Learned 

While all three researchers agree that the research went well and were pleased 
with the outcomes, there were lessons learned. Rick identified a desire to have pushed for 
a better method to reconcile and report differences in coding while it was fresh on the 
minds of the researchers instead of waiting until later to complete the process. He also 
wished that 30 interviews had been conducted, as this larger number may have identified 
“those less frequent responses and themes that may have not yet emerged” even though a 
point of saturation had occurred with only 14 interviews. Including more women in the 
sample would also have been advisable, since the whole logistical domain appears to 
arise out of the concerns of women. 

The process of working separately from the other researchers turned out to be a 
positive experience. We worked on the same project, but separately. We each did our 
parts in isolation from one another. This seemed to work well. In fact, it afforded us the 
opportunity to put together truly different ideas and codes from the data. With more 
interaction throughout the study, we may have missed some of the significant differences 
that were apparent in our working separately. Conjoint work oftentimes blurs differences 
that exist among individuals. 


Concluding Remarks 

In this world of immediate gratification and an underwritten sense that a speedy 
journey is superior to a slower, more reflective one, this study has taught us all in many 
ways. It has been frustrating to all three researchers that we have taken this process so 
slowly and written this article in such a way that we have had to meet many times after 
the completion of the research to reflect on our process of many months ago. This study 
and the subsequent written part of this process have taught us about reflection, merging 
of three different ways of thinking about approaches to research, and patience with the 
process. It is one thing to imagine goals for research and design an appropriate study. It is 
quite another to bring together three very different researchers and work together, yet 
separately, to complete a project that had never been attempted before. On the surface, it 
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appeared that the rewarding elements of this study were giving power to otherwise 
silenced voices in research or possibly the creating of a scale to assist treatment providers 
and their clients. However, after so much reflection and attention to details of the actual 
process of conducting the research, it has become evident that our process is another 
rewarding element of the study. This article serves as a testimony to that rewarding 
process with value we wish to transmit to the readers. After reading this manuscript, we 
hope that the readers will now have more of a transparent understanding of the process of 
using qualitative methods to create instruments for research as well as an enriched view 
of the value of this collaborative work. Furthermore, it is recommended that replication 
of a triangular mixed-methods approach to constructing scales be implemented. Findings 
from this study indicate that using a qualitative interview strategy with three independent 
analytic approaches can have a profound impact on the development of a scale. 
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